RFM analysis

Background

From the Pareto Principle, 80% of sales come from 20% of customers. Usually your business is largely supported by a fraction of your customer base - your best customers.

From a marketing perspective, it is important to understand the characteristics and preferences of your best customers for at least 2 reasons:

  1. to continue to provide the group with what they are looking for and keep them as customers
  2. to target your marketing efforts towards prospects who resemble your best customers

Objective

To identify the best customers who will be loyal from the start

Approach

Perform customer segmentation using RFM clustering to identify the best customers

Idea behind RFM:

R - Recency : Customers who have purchased from your store recently are more likely to convert again than those who haven't visit your store for a while

F - Frequency : Customers who buy from your store more often are more likely to buy again than those who buy infrequenctly

M - Monetary : Customers who spend more are more likely to buy again than those who spend less

Steps

  1. Load the dataset and proceed data cleaning

  2. K-means Clustering to create the Recency, Frequency and Monetary scores for each customer

    Find out the optimal no of clusters to be applied for each attribute. Aim to assign same no of clusters for each attribute, a higher cluster no (score) should reflect a better result,i.e. cluster 3 should be better than cluster 1.

    • Recency: number of days since most recent purchase date (More #days would have lower cluster#)
    • Frequency: number of transactions within the period (higher freq. would have higher cluster#)
    • Monetary: total sales attributed to the customer (higher sales would have higher cluster#)
  3. Compare the RFM clusters (observe the relationship against each other) and calculate the total score

  4. Segment the customers into different categories based on the RFM scores and total score:
    • Champions: our best customers!
    • Loyalists
    • New Customers
    • Can’t Lose Them
    • Slipping
  5. Conclusion on RFM analysis

About the dataset

This Online Retail II data set contains all the transactions occurring for a UK-based and registered, non-store online retail between 01/12/2009 and 09/12/2011.The company mainly sells unique all-occasion gift-ware. Many customers of the company are wholesalers.

Column Descriptors

Data Source: https://www.kaggle.com/mathchi/online-retail-ii-data-set-from-ml-repository

Reference

1. Load the dataset

Handle Missing values

2. K-means Clustering to create the Recency, Frequency and Monetary scores

Recency

We need to find out the most recent purchase date of each customer and check out how many days since that purchase. Then, we can apply K-means clustering to assign each customer a recency score.

Average Recency is 90 days and median Recency is 51 days.

Most are recent shoppers, less than 100 days.

K-means Clustering to assign recency score

The goal of K-means clustering is to group data points into distinct non-overlapping subgroups. The mathematics behind K-means clustering, involves minimizing the sum of square of distances sse between the cluster centroid and its associated data points.

K-means clustering could be used to examine multiple variables, but in this exercise I just focus on 1 variable (R/F/M) each time.

The Elbow Method

The sse(sum of squares) value decreases as number of clusters increases, we aim to get an optimal point after which the reduction of sse is only minimal. In this case, number of clusters = 3 would be the optimal point or we can also choose 4 which resulted in even lower sse.

Cluster 3 is the best with most recent visits, and cluster 0 is the worst.

Frequency

We need to find out no of transaction for each customer which would be the frequency. Then, we can apply K-means clustering to assign each customer a frequency score.

Average Frequency is 95 transactions per customer and median Frequency is 44 transactions.

Frequncy dropped sharpy from count around 100.

K-means Clustering to assign frequency score

Similarly, apply no of clusters = 4 for K-Means clustering and predict the cluster number for each custemer.

Similarly, cluster 3 is the best with most frequent visits, and cluster 0 is the worst.

Monetary

We will calculate the total sales generated by each customer. Then, we will apply K-means clustering to assign each customer a Monetary score.

Average monetary value is 1905 per customer and median monetary value is 656. Some customers carried negative transaction value too.

Most are spending less than 1000.

K-means Clustering to assign Monetary score

Similarly, apply no of clusters = 4 for K-Means clustering and predict the cluster number for each custemer.

3. Compare the RFM clusters

Frequency and Monetary Clusters are dominated with 0 score, while Recency cluster score distribution is rather more evenly distributed.

Overall RFM Score

We have created Recency, Frequency and Monetary scores for each customer, we can sum up them to give a total score. The higher the score, the more valuable the customer is.

5. RFM segmentation

As shown in above graphs, for the given dataset, the Recency, Frequency and Monetary absolute value distribution skewed towards right, which means most customers visited less frequent and consumed less amount but quite a lot are recent visitors. We have most customers filled in the low Frequency & Monetary clusters, while recency clusters are rather evenly distributed and total score of most customers is no more than 3.

With reference of above summary, we can group the customers into 5 segments based on the individual RFM scores and total score:

There are only 57 customers in Champions group with RFM means values at R=7, Frequncy=1115, Monetary=41468. Following that we have the loyalists who would be the potential champions visiting quite recently with moderate spening amount. Then the new customers are whom we would want to turn into loyal customers. The 'can't lost them' and slipping groups are those who are slipping away, we would need retention strategies to keep them.

Most of the customers are newly boarding ones.

Champions visited recently, most frequently and were high spenders. The slipping group had not visited the store for long, more than 200 days (median), and spent a little.

Bubble size represents frequency, the champions group climb along monetary axis with almost 0 recency and big bubble size.

3-D plot shows the distribution of customer segment more clearly.

Conclusion on RFM analysis

Once we are done calculating the RFM scores, we are able to segment the customer pool and identify the best customers - the champion segment. Then we can start to analyse the characteristics and purchasing behavior of this group to figure out why they perceive more value in our business than the folks who seldom visit and consume less amount.

This will help us sharpen the understanding of the target market and be more precise in communicating with actual and potential customers.